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ABSTRACT 


Least  squares  solutions  of  Fredholm  and  Volterra  equations  of  the  first 
and  second  kinds  are  studied  using  generalized  inverses.  The  method  of 
successive  approximations,  the  steepest  descent  and  the  conjugate  gradient  methods 
are  shown  to  converge  to  a  least  squares  solution  or  to  the  least  squares  solution 
of  minimal  norm,  both  for  integral  equations  of  the  first  and  second  kinds. 

An  iterative  method  for  matrices  due  to  Cimmino  is  generalized  to  integral 
equations  of  the  first  kind  and  its  convergence  to  the  least  squares  solution 


of  minimal  norm  is  established. 


ITERATIVE  METHODS  TOR  BEST  APPROXIMATE  SOLUTIONS  OF  LINEAR 
INTEGRAL  EQUATIONS  OF  THE  FIRST  AND  SECOND  KINDS 

W.  J.  Karnmerer^  and  M.  Z.  Nashed 

1.  Introduction. 

Linear  integral  equations  of  the  first  and  second  kinds  that  have  non- 
unique  solutions  or  that  have  no  solution  at  all  arise  in  many  settings.  Physical 
problems  may  lead  to  such  situations  directly  as  in  the  case  of  the  integral 
equation  formulation  of  the  interior  Neumann  problem  for  the  Laplacian  on  a  simply 
connected  region  with  a  smooth  boundary  (see,  for  instance,  [13;  pp.  341-344]). 

On  the  other  hand,  one  is  led  to  such  situations  via  eigenvalue  problems,  as  in 
the  case  of  a  nonhomogeneous  integral  equation  of  the  second  kind  when  the 
associated  homogeneous  equation  has  a  nontrivial  solution.  In  this  case,  if  the 
prescribed  function  appearing  in  the  integral  equation  satisfies  the  compatability 
condition  of  the  Fredholm  alternative,  then  we  have  an  infinite  number  of  solutions; 
otherwise  we  have  no  solution. 

There  are  a  number  of  cases  in  which  one  would  like  to  find  the  solution  of 
minimal  norm  to  a  non-uniquely  solvable  Fredholm  or  Volterra  integral  equation,  or 
to  seek  least  squares  solutions  when  the  integral  equation  in  question  does  not  have 
a  solution  in  the  classical  sense. 

The  bulk  of  the  work  on  iterative  methods  for  solution  of  linear  integral  equations 

is  devoted  to  equations  which  have  unique  solutions  (see  [48],  [36],  [34],  [28],  [31];  these 

references  also  contain  relevant  bibliographies).  The  main  purpose  of  the  present  paper  is 
to  investigate  best  approximate  solutions,  i.  e.  solutions  in  the  sense  of  least 

squares,  and  to  establish  the  convergence  of  the  method  of  successive  approxi  - 

mations,  the  steepest  descent  and  the  conjugate  gradient  methods  to  best  approxi- 
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mate  solutions  of  integral  equations  of  the  first  and  second  kinds.  We  also  generalize 
to  integral  equations  of  the  first  kind  an  iterative  method  for  matrices  due  to 
Cimmino  [5]  and  establish  its  convergence  to  a  best  approximate  solution  of  minimal 
norm. 

It  is  well  known  that  the  numerical  solution  of  integral  equations  of  the 
first  kind  leads  to  difficulties  both  for  the  Fredholm  and  the  Volterra  equations,  since 
the  solution  does  not  depend  continuously  on  the  data.  This  instability  of  the 
integral  equation  of  the  first  kind  also  carries  over  to  the  solutions  of  the  algebraic 
system  arising  from  discretization  of  the  integral  equation.  Continuous  dependence 
of  solutions  on  the  data  can  be  brought  about  by  taking  the  notion  of  a  "solution"  to 
mean  a  "least  squares  solution"  and  by  restricting  the  class  of  admissible  solutions 
in  a  suitable  way,  for  instance  by  constraining  the  solution  to  have  an  a  priori 
bound.  Thus  the  notion  of  a  least  squares  solution  is  part  of  a  natural  setting 
for  ill-posed  problems  and  lends  itself  more  readily  to  mathematical  programming, 

(see  [8]).  Moment  discretizations  and  related  aspects  of  smoothness  and  regular¬ 
ization  in  least  squares  solutions  will  be  examined  using  generalized  inverses 
in  a  subsequent  note. 

A  numerical  approach  to  obtain  an  approximate  solution  to  a  non-uniquely 
solvable  Fredholm  integral  equation  of  the  second  kind  has  been  studied  recently  by 
Atkinson  [1].  First  the  equation  is  recast  as  a  new  uniquely  solvable  equation  and 
then  the  integral  operator  is  approximated  using  numerical  integration.  Our  approach, 
in  contrast,  is  iterative,  and  we  do  not  stipulate  that  the  equation  is  solvable  since 
we  seek  solutions  in  the  least  squares  sense.  We  carry  parallel  developments  for 
the  integral  equations  of  the  first  and  second  kinds,  using  generalized  inverses  as  a 
basic  tool  throughout. 
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Generalized  Inverses  and  Linear  Operator  Equations  of  the  First  and  Second 


Kinds. 

Let  X  and  Y  be  two  Hilbert  spaces  over  the  same  (real  or  complex)  scalars, 
and  let  T  be  a  bounded  linear  operator  on  X  into  Y,  whose  range  is  not  neces¬ 
sarily  closed.  We  denote  the  range  of  T  by  R(T),  the  null  space  of  T  by  V(T) 

*  X 

and  the  adjoint  of  T  by  T  .  For  any  subspace  S  of  X  or  Y,  we  denote  by  S 
the  orthogonal  complement  of  S  and  by  S  the  closure  of  S.  Then  the  following 
relations  hold  (see,  for  instance,  Yosida  [50]) 


X  -  M(T)©W(T)1, 


Y  =  n(t*)©w(tV, 


R(T)  *=  N(T*)\  R(T*)  -  W(T)A, 


and  the  restriction  of  T  to  fj(T)'L,  denoted  by  T|w(T)X,  has  an  inverse  which  is 
necessarily  j. 

not^continuous .  Let  P  denote  the  orthogonal  projection  of  X  onto  N (T)  and  let 

it  ± 

Q  denote  the  orthogonal  projection  of  Y  onto  M(T  )  .  then 


Tx  -  TPx 


for  all  x  e  X, 


*  * 

T  y  *  T  Qy 


for  all  y  e  Y. 


We  associate  with  the  linear  operator  equation 


Tx  -  y  for  y  e  Y, 


the  projectional  equation 
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(2.7) 


Tx  »  Qy 


for  y  e  Y. 


Obviously,  (2.7)  is  different  from  (2.6)  only  if  R(T)  is  not  dense  in  Y,  in 
which  case  the  solvability  of  (2.7)  does  not  necessarily  imply  the  solvability 
of  (2.6).  For  any  y  e  R(T) (+)  R(T)1,  (2.7)  is  solvable  and  the  set  S  of  all 
solutions  is  a  closed  convex  subset  of  X;  hence  it  contains  a  unique  element  of 
minimal  norm.  In  this  manner,  we  get  a  mapping  which  assigns  to  each  y  e  R(T)(+) 
RW*",  the  unique  solution  of  minimal  norm  of  (2.7).  We  call  this  mapping  the 
generalized  inverse  of  T  and  denote  it  by  T*.  We  formalize  this  in  the  follow¬ 
ing  definition. 

Definition  2.1.  Let  T  :  X  H  Y  be  a  bounded  linear  operator.  The  generalized 
inverse  T*  of  T  is  the  mapping  whose  domain  is  P(T^)  *  R(T)  ©R(T)J‘,  and  T^y  = 
v  e  X,  where  vcS  =  {xeX:Tx*>  Qy}  and  ||v||  <  ||u|j  for  all  u  e  S,  u  /  v. 

t  - 1  t 

Note  that  N(T  )  =  W(Q)  «  R(T)  and  that  T  is  linear  but  not  necessarily 
bounded. 

A  natural  question  that  arises  is:  What  is  the  significance  of  a  solution 
of  (2.7)  as  far  as  (2.6)  is  concerned?  To  this  end,  we  state  the  following 
proposition. 

Proposition  2.1.  For  a  fixed  y  e  Y,  let  S  ■  {x  6  X  :  Tx  “  Qy}  and  N  <=  {x  e  X  : 

*  *  , 

T  Tx  ■  T  yj .  Then  5  =  N. 

*  *  * 

Proof.  Let  u  e  S,  then  TTu«TQy»Tyby  (2.5).  Conversely,  if  v  e  N,  then 

*  *  *  ii  * 

T  Tv  »  T  y  *  T  Qy,  which  means  that  Tv  -  Qy  e  N(T  ).  But  Tv  -  Qy  is  also  in 
R(T) .  Therefore,  Tv  -  Qy  €  N(T*)  0  R(T)  -  {o}. 

Definition  2.2.  An  element  u  e  X  is  called  a  least  squares  solution  of  the 
linear  operator  equation  (2.6)  if 
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||Tu  -  y||  =  inf  {||Tx  -  y||  :  x  e  x} . 


An  element  x  is  called  a  least  squares  solution  of  minimal  norm  of  (2.6)  if  x 
is  a  least  squares  solution  of  (2.6)  and  ||x||  s  ||u||  for  all  least  squares  solutions 
u  of  (2.6) . 

For  a  given  y  e  Y,  the  set  of  all  least  squares  solutions  of  (2.6)  coincides 
with  the  set  N  of  all  solutions  of  the  "normal"  equation  T*Tx  =  T*y. 

Combining  this  observation  with  Proposition  2.1  and  Definition  2.1,  we  arrive 
at  the  following  proposition. 

Proposition  2.2.  Let  T  be  a  bounded  linear  operator  on  X  into  Y,  whose  range 
is  not  necessarily  closed.  The  least  squares  solution  x  of  minimal  norm  of  the 
linear  operator  equation 


Tx  =  y  ye  D(T^) 

_  t  t 

is  given  by  x  =  T  y.  The  set  of  all  least  squares  solutions  for  each  y  e  D(T  ) 

is  given  by  T  y©N(T). 

It  follows  easily  from  the  definitions  of  P,  Q,  and  T^  that  P  *  T^T,  Q  is 
the  continuous  linear  extension  of  TT^  to  Y,  and  T^  Er(T)  J  *  (o). 

If  the  range  of  T  is  closed,  then  by  the  closed  range  theorem  (see  Yosida 
[50]),  the  range  of  T  is  also  closed  and  one  obtains  from  (2.3)  and  (2.2), 

R(T)  *  W(T*)X,  R(T*)  =  N(T)1,  and  Y  ■  R(T)  ©R(T)1.  The  generalized  inverse 
in  this  case  is  defined  on  all  of  Y  and  is  also  bounded,  and  Q  -  TT^.  The 
following  proposition  sheds  more  light  on  the  generalized  inverse  of  a  bounded 
linear  operator  with  closed  range;  see  also  [ 33] . 
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Proposition  2.3.  For  a  bounded  linear  operator  T  :  X  -  Y,  the  following 
statements  are  equivalent: 

(a)  T  has  a  bounded  generalized  inverse. 

(b)  T  has  a  bounded  right  inverse. 

(c)  The  restriction  of  T  to  W(T )A  has  a  bounded  inverse. 

(d)  The  operator  {t|M(T)‘L]  1  has  a  bounded  linear  extension  to  all  of  Y 
so  that  its  null  space  is  W(T  ). 

(e)  The  quotient  space  X/W(T)  is  isomorphic  with  R(T). 

(f)  The  number  y(T)  defined  by  y(T)  =  SUP  {inf  [||x||  :  Tx  =  y}  :  y 'e  R(T), 

||y||  =  l}  is  finite. 

(g)  R(T^  is  closed. 

(h)  T  is  normally  solvable  in  the  sense  of  Hausdorff,  i.e.  for  a  given 

y  e  Y,  the  necessary  and  sufficient  condition  for  the  equation  Tx  =  y 
to  be  solvable  is  that  y  e  NCT*)1. 

(i)  inf  { ||t.:  -  y||  :  x  e  X}  is  attained  for  each  y  e  Y. 

(j)  All  pseudoinverses  of  T  are  bounded.  (A  linear  operator  M  with  the 
property  that  TMT  =  T  is  called  a  pseudoinverse  of  T. ) 

(k)  There  exists  a  unique  operator  T^  :  Y  -*  X  such  that  T^TT^  «  T  ,  T^T  =  P, 
and  TT^  =  Q. 

Remark  2.1.  The  concept  of  the  generalized  inverse  of  a  linear  operator 
plays  a  decisive  role  in  the  study  of  convergence  of  iterative  methods  for 
integral  equations  that  are  considered  in  the  present  paper  It  is  appropriate 
here  to  point  out  that  historically  notions  of  generalized  or  pseudo  inverses 
appeared  first  in  the  context  of  analysis,  rather  than  in  the  setting  of  matrices 
and  algebraic  problems.  More  specifically,  the  germ  of  these  notions  may  be 
found  in  the  celebrated  paper  of  Fredholm  [lO],  in  the  work  of  Hurwitz  [20]  on 
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pseudo-resolvents,  the  work  of  Hilbert  [18],  Elliott  [.9],  Reid  [37],  and  others 
on  generalized  Green's  functions  (see  [39]  for  other  references  and  a  detailed 
discussion  of  this  aspect  of  the  history  of  generalized  inverses). 

The  classical  development  of  integral  equations  during  the  first  half  of 
this  century  is  rich  with  instances  of  implicit  ideas  and  concepts  which  led  to 
fundamental  abstractions  in  the  setting  of  functional  analysis.  Many  aspects 
of  linear  and  nonlinear  operator  theory  (for  example,  compact  operators,  spectral 
theory  of  Riesz-Schauder,  gradient  mappings,  monotone  operators,  collectively 
compact  operators,  etc.)  owe  their  origin  or  motivation  to  considerations  of 
integral  equations.  In  turn,  when  the  various  aspects  of  the  theory  of  integral 
equations  are  re-examined  in  the  light  of  such  results  of  operator  theory, 
usually  new  insight  and  results  are  obtained 

The  operator- theoretic  framework  in  terms  of  generalized  inverses  is  a 
natural  setting  for  integral  equations  which  have  no  solutions  in  the  classical 
sense,  or  which  have  non-unique  solutions.  Generalized  inverses  have  been 
recently  used  in  connection  with  Green's  functions  and  boundary  value  problems 
and  other  aspects  of  ordinary  and  partial  differential  equations  by  Reid  [38], 
[39],  Loud  [29],  [30],  Wyler  [49],  Landesman  [27],  Kallina  [2l],  Halany  and 
Moro  [14]  and  others. 

Remark  2.2.  The  earliest  work  explicitly  devoted  to  the  study  of 
generalized  inverses  of  linear  operators  is  that  of  Tseng  [44],  [45],  [46] 

(see  also  [2]  for  a  summary  of  some  of  Tseng's  results  and  a  comprehensive 
development  of  some  aspects  of  generalized  inverses,  together  with  an  extensive 
bibliography).  Pseudoinverses  of  linear  operators  were  also  defined  and  used 
by  Friedrichs  [12],  Hamburger  [l5],  and  Sheffield  [40].  The  generalized  inverse 
of  a  continuous  linear  transformation  with  closed  range  has  been  studied  by 
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Desoer  and  Whalen  [7],  Beutler  [3],  Petryshyn  [35],  Sheffield  [40],  Votruba  [47] 
and  others .  Various  (not  necessarily  equivalent)  definitions  of  generalized 
inverses  for  the  case  when  the  range  of  the  continuous  operator  is  not  closed 
as  well  as  for  unbounded  closed  operators  have  been  proposed  by  Tseng  [45],  [46], 
Hestenes  [16],  Beutler  [3],  and  others.  Pseudoinverses  for  closed  operators 
with  closed  range  have  been  briefly  considered  also  by  Wyler  [49]  and  Reid  [39]. 

The  definition  of  a  generalized  inverse  that  was  introduced  in  this  section 
is  equivalent  in  the  case  when  R(T)  is  closed  to  the  definitions  in  [7],  [3], 
[35].  It  has  the  advantages  of  focusing  on  the  problem  of  solvability  of  the 
operator  equation  and  of  treating  the  cases  when  R(T)  is  closed  and  when  R(T)  is 
not  closed  in  the  same  framework,  thereby  exhibiting  the  distinctive  features 
of  these  two  cases  as  far  as  least  squares  solutions  are  concerned. 

In  Proposition  2.3  we  stated  several  characterizations  of  a  bounded  linear 
operator  with  closed  range.  The  following  proposition  exhibits  specific  classes 
of  such  operators,  which  will  be  used  in  the  analysis  of  integral  operators  in 
the  following  sections. 

For  any  linear  operator  T  :  X  -•  X, 

{0}  c  W(x)  C  W(r,>  c  ••*  c  W(T^)  c  ... 

and 

X  3  l!(T)  3  R(Ta)  3  ...  3  RO*)  3  ...  . 

We  recall  that  T  is  said  to  have  a  finite  ascent  if  for  some  nonnegative  integer 

IT  |  |  L 

r,  "(T  )  =  N(T  )  for  k  «*  1,2 .  In  this  case,  the  smallest  such  integer 

is  called  the  ascent  of  T.  The  descent  of  T  is  similarly  defined  as  the  smallest 
integer  r  for  which  R(Tr)  -  R(Tr+k)  for  k  ■  1,2,  ...  ,  assuming  T  is  of  finite 
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ascent.  If  T  has  a  finite  descent  6  and  a  finite  ascent  a,  then  a  =>  6  and  X  = 
R(TPr)©W(Tpr)  (see  [43]).  In  connection  with  integral  operators,  we  note  that 
for  every  compact  linear  operator  A  on  a  normed  linear  space  X  into  X,  the 
operator  XI  -  A  has  a  finite  ascent  and  descent  if  X  ^  0.  Also,  if  T  is  a 
normal  operator,  i.e.  T*  T  =  TT*,  the  ascent  is  either  0  or  1  (see  [43], 

Theorem  6.2  F) . 


Proposition  2.4.  The  set  of  all  bounded  linear  operators  with  closed  range 
includes  the  following  classes  of  operators: 

(a)  all  operators  which  are  bounded  below,  i.e.,  ||Tx||  S  m  |jx||  ,  m  >  0,  for  all 
x  e  X; 

(b)  all  operators  of  the  form  T  =  Tx  +  T2>  where  R(T>)  is  closed  and  RCig) 
is  finite  dimensional; 

(c)  all  operators  of  the  form  T  =  A  -  XL,  X  ^  0  where  A  is  completely 
continuous  (i.e.,  maps  each  bounded  set  into  a  compact  set)  and  L  has 
a  bounded  inverse; 

(d)  all  continuous  normal  operators  of  finite  descent. 


Proof.  Parts  (a)  and  (b)  are  obvious.  To  prove  (c)  we  recall  [51 3  that  for  any 
c  >  0,  a  completely  continuous  operator  A  can  be  decomposed  in  the  form  A  =  4- 
Ag,  where  JlA^  ]|  <  e  and  R(A3)  is  finite  dimensional.  Take  e  =  _ Then  T  «* 


(Ax  -  XL)+A2  and  <  1.  This  implies  that  Al  -  XL  is  invertible  on  all 

of  Y  and  hence  has  a  closed  range.  Thus  T  has  a  closed  range  by  part  (b) . 

(d)  Since  T  is  assumed  to  be  normal  and  of  finite  descent,  its  descent  must  be 


either  zero  or  one.  If  the  descent  is  zero,  then  R(T)  =  X  and  hence  closed. 
Suppose  now  that  the  descent  of  T  is  one,  then  X  =  R(T)(+)N(T).  But  we  also 
have  X  -  W(T)  ©N(T)i  since  T  is  continuous.  Thus  R (T)  is  closed  if  and  only  if 
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R(T)  =  W(T)X.  Let  x  e  W(T)X,  then  x-Tu+z,  ueX,  ze  W(T),  and  <x,  v)  *  0 

jc  jl 

for  all  v  e  W(T) .  But  <Tu,  v>  ■  <u.  Tv)  *  0  for  v  e  M(T),  since  N(T)  ■  N(T  ) 
for  any  normal  operator.  This  Implies  that  (z,  v)  =  0  for  all  v  e  N(T).  Setting 
v  «*  z,  we  get  z  =  0.  Hence  x  -  Tu  +  z  ■  Tu,  i.e.  u  e  R(T).  This  proves  that 
W(T)X  c  R(T).  The  inclusion  R(T)  c  N(T)X  is  obvious. 

Proposition  2.5.  A  completely  continuous  linear  operator  T  :  X  -*  Y  does  not 
have  a  closed  range  unless  R(T)  is  finite  dimensional. 

Proof.  Suppose  T  is  completely  continuous  and  R(T)  is  closed.  Then  T  has  a 
bounded  generalized  inverse  T^  defined  on  all  of  Y.  Thus  TT^  being  the  compo¬ 
sition  of  a  completely  continuous  operator  and  a  bounded  operator  is  completely 
continuous.  On  the  other  hand,  we  have  TT^  *=  Q.  Thus  q|R(T)  =  l|R(T)  is 
completely  continuous,  which  implies  that  R(T)  is  finite  dimensional. 

We  recall  also  that  the  range  of  a  completely  continuous  linear  operator  T 
is  always  separable  and  that  a  (T) ,  the  point  spectrum  of  T,  contains  at  most  a 
countable  set  of  points  with  zero  the  only  possible  accumulation  point. 

We  now  consider  the  linear  operator  equations 

(2.8)  Ax  -  Xx  =  y 

(2.9)  Ax  -  y 

where  y  is  a  given  element  in  a  Hilbert  space  H  and  A  is  a  completely  continuous 
linear  operator  on  H  into  H.  (2.9)  and  (2.8)  are  usually  referred  to  as  equations 
of  the  first  and  second  kinds,  respectively,  by  analogy  with  integral  equations. 
For  any  X  /  0,  R(A  -  XI)  is  closed  and  we  have  from  (2.1)  -  (2.3), 
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(2.10) 


H  -  R(A  -  XI) ©  W(A*  -  Xl) 


»  R(A*-  XI)©N(A  -  XI),  X/0. 

The  criterion  for  solvability  of  (2.8)  can  be  completely  analyzed  using  the 
well  known  theorems  of  Fredholm  -  Riesz ,  which  are  based  on  (2.10),  and  the 
following  relations  for  X  J-  0  (see,  for  instance,  [ 50 ] ,  [ 51 ]) : 

dim  W(A  -  XI)  =  dim  W(A*  -  XI)  <  « 

dim  R(A  -  XI)  =  dim  R(A*  -  XI) 

It  then  follows  that  for  a  given  X  f  0,  (2.8)  has  a  solution  for  all  y  e  H  if 
and  only  if  W(A  -  Xl)  =  {o}.  On  the  other  hand,  (2.8)  has  a  solution  for  a 

k  — 

®iven  y  e  H  and  X  /  0  if  and  only  if  y  is  orthogonal  to  W(A  -  XI).  If  X  f  0 
is  not  an  eigenvalue  of  the  operator  A,  then  (A  -  Xl)  1  is  bounded  and  R(A  -  XI)  * 
H.  If  X  ^  0  is  an  eigenvalue  of  A,  then  R(A  -  Xl)  is  a  closed  proper  subspace 
of  H. 

.  *  I 

For  the  solvability  of  (2.9)  for  a  given  y  e  H,  the  condition  y  e  W(A  )■*• 
is  necessary  but  not  sufficient,  since  the  range  of  A  is  not  closed  unless  it 
is  finite  dimensional  (Proposition  2.5).  The  alternative  theorem  does  not  hold, 

k 

and  one  does  not  get  a  decomposition  theorem  of  H  in  terms  of  R (A)  and  W (A  ) . 

*  - 

On  the  other  hand,  one  can  determine  the  additional  requirement  that  y  e  N  (A  )■*■ 

must  satisfy  in  order  for  (2.9)  to  be  solvable,  in  terms  of  the  eigenvalues 

k 

and  the  orthonormal  eigenvectors  {0^}  of  the  operator  AA  ,  namely. 
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(2.11) 


<  “. 


#n>|3 
n 

(See,  for  Instance,  [6],  [42]. ) 

In  the  present  paper,  we  are  primarily  interested  in  the  case  when  the  above 
solvability  criteria  are  not  satisfied,  so  that  (2.8)  and  (2.9)  do  not  have 
solutions,  and  also  in  the  case  when  these  equations  have  an  infinite  number  of 
solutions.  We  are  interested  in  "best  approximate  solutions"  of  these  equations 
or,  more  precisely,  least  squares  solutions  of  minimal  norm  in  the  sense  of 
Definition  2.2.  The  following  theorem  follows  easily  from  the  preceding  propo¬ 
sitions. 

Theorem  2.1.  Let  A  be  a  completely  continuous  linear  operator  on  a  Hilbert 
space  H  into  H. 

(a)  For  each  X  ^  0,  the  operator  A  -  XI  has  a  bounded  generalised  inverse 

f  A  t 

(A  -  Xl)  defined  on  all  of  H  and  x  *  (A  -  XI)  y  is  the  unique  best  approximate 

solution  of  minimal  norm  of  (2.8)  for  each  y  e  H,  i.e., 

inf  {||  (A  -  Xl)x  -  y||  :  x  e  H}  =  ||  (A  -  Xl)x  -  y|| 

and  ||x||  <  ||u||  for  all  u  such  that 

||  (A  -  Xl)x  -  y||  -  ||  (A  -  Xl)u  -  y||,  u  ^  x. 

In  particular  if  X  ^  0  and  y  e  R(A  -  XI),  then  x  =>  (A  -  Xl)*y  is  the  unique 
solution'  of  (2.8)  of  minimal  norm.  If  X  /  0  it  in  the  resolvent  of  A,  then 
(A  -  Xl)^  =  (A  -  XI)  1  and  (2.8)  has  a  unique  solution  for  each  y  c  H. 
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* 

(b)  The  operator  A  has  a  generalised  inverse  A  defined  on  R(A)©R(A)A. 

A  is  unbounded  unless  R(A)  is  finite  dimensional.  The  linear  operator  equation 
(2.9)  has  a  unique  least  squares  solution  for  each  y  e  C(A^).  If  also  y  e  W(A*)a 
and  (2.11)  holds,  then  x  =  A^y  is  the  unique  solution  of  (2.9)  of  minimal  norm. 

(c)  The  set  of  all  least  squares  solutions  of  (2.8)  for  X  /  0  is  given  by 
(A  -  Xl)%©N(A  -  XI)  for  each  y  e  H.  The  set  of  all  least  squares  solutions 
of  (2.9)  for  each  y  e  R(A)  ©R(A)i  is  given  by  A^y@W(A). 

Remark  2.3.  The  operators  P  and  Q  played  a  crucial  role  in  the  definition  and 
development  of  a  generalized  inverse  of  a  bounded  linear  operator  between  two 
Hilbert  spaces.  The  definition  can  be  extended  easily  to  Banach  spaces.  We 
consider  for  instance  the  case  of  a  continuous  linear  operator  T  :  X  -*  Y,  where 
X  and  Y  are  Banach  spaces  over  the  real  or  complex  numbers,  and  T  has  a  closed 
range.  Let  and  P^  be  given  projectors  onto  M(T)  and  R(T)  respectively.  (By 
a  projector  P  we  mean  as  usual  a  continuous  linear  and  idempotent  (1^  =  P) 
operator.)  The  unique  bounded  linear  operator  T^  (which  depends  on  P^  and  P^) 
of  Y  into  X  satisfying  TtTT+  =  T+,  TfT  =  I  -  P  ,  and  TTf  =  P2  is  called  the 
generalized  inverse  of  T  relative  to  the  projectors  P^  and  P2>  In  the  case  of 
Hilbert  spaces  we  have  chosen  P^  and  P2  to  be  the  orthogonal  (equivalently  self- 
adjoint)  projectors.  Although  other  choices  are  possible,  they  do  not  lead  to 
the  desirable  connection  with  least  squares  solutions  stated  in  Proposition  2.2. 

Finally,  we  remark  that  in  the  case  of  a  generalized  inverse  on  a  Banach 
space  (whose  norm  is  not  induced  by  an  inner  product),  T^y  is  not  necessarily 
a  best  approximate  solution  of  Tx  ■=  y  for  y  «  p(T^). 
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3.  Best  Approximate  Solutlona  of  Fredholm  and  Volterra  Linear  Integral  Equations 
of  the  First  and  Second  Kind*. 

Throughout  this  section,  the  kernel  K(a,  t)  Is  a  function  In  b]  x 

[a,  b]} t  1*6* , 


jtj.  !*<••  ‘>1* 41  at<*  • 

and  y  Is  a  given  element  In  I^Ca,  b]  with  the  usual  inner  product.  For  simplicity 
of  notation,  we  take  K(s,  t)  to  be  real;  all  the  results  hold  for  complex  kernels 
as  well  with  obvious  modifications.  Let  ft  and  C  denote  respectively  the  Fredholm 
and  Vol terra  linear  integral  operators 

(3.1)  ”  Ja  K**’  X(t)  dt 

and 

(3.2)  55*  =  Ja  K(‘,  t)  x(t)  dt  . 

ft  and  3?  map  L^a,  b]  into  Itself  and  are  completely  continuous.  We  consider  the 
Fredholm  equations  of  the  first  and  second  kinds 

(3.3)  -  y  . 

(3.4)  x  -  XRx-  y  , 

and  the  Volterra  equations  of  the  first  and  second  kinds 
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(3.5) 


S*  -  y 


(3.6) 


x  -  X<p  x  =  y  . 


A  function  u  e  LjCa,  b]  is  called  a  best  approximate  solution  of  (3.3)  if 
it  minimizes  the  functional  ||ftx  -  y||  in  the  I^-norm.  An  element  is  a  best 
approximate  solution  if  and  only  if 


(3.7) 


*  * 
ft  ftu  =  ft  y 


where 


ft  x  =  K(t, • )  x(t)  dt  . 

a 


Equation  (3.7)  can  be  put  in  the  form 


rb 

(3.8)  Ja  M(s,  t)  x(t)  dt  =  yj  , 


where  H(s,  t)  =  K(r,  s)  K(r,  t)  dr,  and  y^(t)  =  K(t,  s)  y(t)  dt.  As  pointed 
out  in  a  more  general  setting  in  Section  2,  (3.8)  need  not  have  solutions  for  all 
y  €  H. 

Similarly  the  problem  of  finding  best  approximate  solutions  of  (3.4)  is 
equivalent  to  solving  the  normal  equation 

(3.9)  (I  -  X(ft  +  ft*)  +  \Vft)x  -  (I  -  Xrf)y, 

or  equivalently, 


(3.10) 


f  x<8>  *  X  J«  k<8’ 


t)  x(t)  dt  =  g(s),  where 


^k(s,  t)  =  K(s,  t)  +K(t,  a)  -  XM(i,  t),  and  g(s)  ■  y(a)  -  Xy1(a) 


#1117 


-15- 


Note  that  the  kernel  k(s,  t)  is  always  symmetric  and  that  X  appears  quadra tlcally 
is  (3.10).  Equation  (3.10)  always  has  solutions. 

It  Is  easy  to  show  that  the  adjoint  of  the  Volterra  operator  S3  Is  given  by 

SB  x  s  K(t,‘)  x(t)  dt  . 

e 

*  * 

Thus  the  normal  equation  5?  Sx  ■  SB  y  takes  the  form 


(3.11) 


f  K(t,  s)  K(t,  r)  x(r)  dr  dt  -  |  K(t,  e)  y(t)  dt  =  0. 
s  J  a  J  8 


Interchanging  the  order  of  integration  in  the  double  integral  in  (3.11)  leads  to 


K(t,  8)  K(t,  r)  x(r)  dr  dt  + 


K(t ,  s)  K(t ,  r)  x(r)  dt  dr 
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where 


N(s,  t) 


{ 


N(s,  t) 
N(t,  s) 


a  s  t  <  8 
s  *  t  <  b  . 


Thus  the  problem  of  finding  the  best  approximate  solution  of  the  Volterra 
integral  equation  (3.5)  is  tantamount  to  finding  the  solution  of  minimal  norm 
of  the  Fredholm  integral  equation  (3.12). 

Finally  the  problem  of  solving  (3,6)  in  the  least  squares  sense  is  equivalent 
to  solving  an  equation  of  the  form  (3.9)  with  ft  replaced  by  SB,  or 


(3.13) 


K(s,  t)  x(t)  dt  - 


+  Xa 


K(t,  s)  K(t ,  r)  x(r)  dr  dt  = 


Xj8  K(t,  s)  x(t)  dt 

rb 

y(s)  -  xjs  K(t ,  s)  y (t)  dt. 


This  is  equivalent  to  the  Fredholm  equation  of  the  second  kind 


(3.14) 

where 


x(s)  -  Xj8  k(s,  t)  x(t)  dt  =  g(s) 

k(s,  t)  =  K(s,  t)  -  XN(s,  t), 


K(s,  t) 


K(s,  t) 

. 

.K(t,  8) 


a  <  t  <  s 
8  <  t  *  b 


and  N(8,  t)  as  before. 
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As  corollaries  to  Theorem  2.1,  we  obtain  existence  and  uniqueness  theorems 
for  the  best  approximate  solution  of  minimal  norm  of  the  Integral  equations  (3.4) 
and  (3.6)  for  any  y  e  l^Ca,  b],  and  of  the  integral  equations  (3.3)  and  (3.5)  for 
any  y  e  R(R)  0R(R)J"and  y  e  R{93)©R(5B)  respectively. 

Theorem  3.1.  Let  K(s,  t)  e  L,{[a,  b]  X  [a,  b]}  and  X  be  a  characteristic  value 

rb 

of  the  Fredholm  integral  operator  (3.1),  i.e.,  for  some  cp  ^  0,  Xl  K(s,  t)  cp(t)dt 
«  cp(s) .  Then  the  generalized  inverse  of  (I  -  XR)  exists  and  is  bounded. 
Furthermore, 

(a)  if  for  a  given  y  c  LjCa.  b]>  (3.4)  is  solvable,  i.e.,  y  e  11(1  -  XR)"1, 

*  t 

then  x  =  (I  -  XR)  y  is  the  unique  solution  with  minimal  Lj-norm; 

(b)  if  for  a  given  y  e  LjC®.  b],  (3.4)  does  not  have  a  solution,  i.e., 

i  —  *  x  *  t 

y  {  W(I  -  XR  )  ,  then  x  =  (I  -  XR)  y  is  the  best  approximate  solution  of  (3.4) 

of  minimal  norm,  i.e.,  it  minimizes  |j  (I  -  XR)x  -  y||  and  has  a  smaller  L^-norm 
than  any  other  u  that  minimizes  ||  (I  -  XR)x  -  y|| . 

Equivalently,  the  normal  operator  equation  (3.9)  is  always  solvable  and 

i k  f 

x  =  (I  -  XR)  y  is  the  unique  solution  of  minimal  norm. 

Similar  results  hold  for  the  Volterra  equation  of  the  second  kind  (3.6), 
and  the  corresponding  normal  equation  (3.14). 

Theorem  3.2.  Let  K(s,  t)  e  L 2 £C a ,  b]  X[a,  b]}. 

(a)  The  generalized  inverse  of  R  exists  on  the  domain  p(R^)  =  R(R)  +R(R)i. 
is  unbounded  unless  R (R)  is  finite  dimensional. 

(b)  If  for  a  given  y  c  LjCa,  b],  (3.3)  is  solvable  in  LjCa,  b],  i.e.,  if 

y  e  W(R*)a  and  ^un  |(y>  <Pn)|s  <  "»  where  are  the  characteristic  values  and 

,  ,  *  f 

{<PnJ  are  the  orthonormal  eigenvectors  of  the  operator  RR  ,  then  R  y  is  the  unique 

solution  with  minimal  L^-norm. 

(c)  If  for  a  given  element  y  e  LjC®.  b],  (3.3)  does  not  have  a  solution, 
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then  x  ■  it  y  is  the  unique  best  approximate  solution  of  (3.3)  for  each  y  e  R(R)  + 

R(R)1.  The  best  approximate  solution  does  not  depend  continuously  on  y  unless 

R ($0  is  finite  dimensional. 

Similar  statements  hold  for  the  operator  5J  and  (3.5). 

As  stated  in  Remark  2.3,  the  generalized  inverse  of  a  bounded  linear  operator 
T  on  a  Banach  space  X  to  a  Banach  space  Y,  which  can  be  defined  relative  to  two 
projectors  P  and  Q,  does  not  possess  the  least  squares  property  stated  in 
Proposition  2.2.  Thus  Theorems  3.1  and  3.2  do  not  extend  to  integral  equations 
on  the  Banach  space  CLa,  b]  with  the  best  approximate  solution  taken  in  the 
sense  of  the  maximum  norm!  However,  we  can  still  study  integral  equations  on 
c[a,  b]  with  the  best  approximate  solution  taken  in  the  sense  of  the  L^'norm. 

The  generalized  inverse  in  the  setting  of  l^Ca,  b]  still  gives  a  best  approximate 
solution.  Furthermore,  the  following  theorem  shows  that  for  each  y  e  c[a,  b], 

(I  -  Ar)%  is  a  continuous  function,  i.e.,  whereas  (I  -  A$^:  LgCa,  b]  -  b], 

the  restriction  of  (I  -  AR)^  to  c[a,  b]  has  its  range  in  c[a,  b]. 

Theorem  3.3.  (I  -  AR)^y  e  c[a,  b]  for  any  y  e  c[a,  b]  and  K(s,  t)  e  c[[a,  b]  X 
Ca,  b]}. 

Proof.  For  K(s,  t)  e  c{[a,  b]  X  [a,  b]},  consider  I  -  AR.  Clearly  R  :  l^ta,  b]  -• 
Cra,  b]  and  R(I  -  AR)  is  closed  in  Ljta,  b].  Hence 

L2Ca,  b]  =  R(I  -  AR)©{R(I  -  Ar}a. 

-  R(I  -  A R)  ©N(I  -  A  ft*}. 


The  given  continuous  function  y  can  be  written  as  y  =  y^  +  y2>  where  y^  ■  (I  -  AR)x 
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for  some  x  e  L^Ca,  b]  and  (I  -  XR)y2  »  0,  i.e.,  y 2  ■  XRy2  which  implies  that 
y^ia  a  continuous  function.  Thus  y^  =  y  -  y^  «  (I  -  XR)x  is  also  continuous. 
How  (I  -  XR)*  (I  -  XR)  a  p,  where  P  is  the  orthogonal  projection  on  W(I  -  Xfl)x, 
and 


L2Ca,  b]  a  W(I  -  XR)©N(I  -  XR)1. 

Thus  we  have  x  =  x^^  +  x^,  where  (I  -  Xff)x^  a  0  and  x2  cW(I  -  XRX1.  This  shows 
that  x^  is  also  continuous  and  hence  so  is  x2.  On  the  other  hand  x2  =  (I  -  X  R)  % 
since  (I  -  XR)x  =  y  and 

(1  -  XR)^  (I  -  XR)  x  a  Px  a  (I  -  Xj&^y  a 


This  completes  the  proof. 

Remark.  In  view  of  Proposition  2.4d,  the  theory  presented  here  also  applies 
to  integral  equations  with  normal  operators.  For  simplicity  we  limit  our 
presentation  to  the  integral  operators  described  in  Theorems  3.1  and  3.2.  The 
modifications  are  simple  in  view  of  the  excellent  exposition  on  normal  operators 
in  the  context  of  Integral  equations  given  in  Zaanen  C 51 ] . 
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.  A  Generalization  to  Integral  Equations  of  a  Method  of  Citnmlno  and  Related 
Aspects  of  the  Successive  Iterative  Method  for  Best  Approximate  Solutions. 

Cinsni.no  [5]  devised  an  iterative  scheme  for  the  solution  of  a  linear  system 

of  equations  Ax  =  y,  where  A  =  (a^)  is  a  square  matrix,  which  converges  even  if 

the  system  of  equations  is  inconsistent,  provided  that  the  rank  of  the  matrix  A 

is  greater  than  one.  (See  Problem  16,  p.  119  in  Householder  C 19] ;  Cimmino's 

method  for  matrices  was  examined  in  the  setting  of  generalized  inverses  by 

Votruba  [473.)  In  this  section  we  generalize  Cimmino's  method  to  integral 

equations  of  the  first  kind  and  prove  its  convergence  to  the  best  approximate 
t  t 

solution  jty,  for  any  y  e  P(R),  provided  R(jO  is  of  dimension  greater  than  one. 

We  also  show  that  the  generalization  of  Cimmino's  method  can  be  recast  as  a 
successive  approximation  method  with  a  specifically  chosen  averaging  parameter, 
and  establish  convergence  theorems  for  best  approximate  solutions  of  integral 
equations  of  the  first  and  second  kinds. 

To  motivate  the  generalization  and  to  place  the  method  in  proper  perspective, 
we  first  discuss  briefly  Cimmino's  method  for  matrices.  Let  r^,  ...,  r^  denote 
the  rows  of  the  matrix  A;  these  rows  determine  n  hyperplanes  in  given  by 

—  (x  *  (r^,  x)  3  y^} ,  i  —  1,  • • « ,  n. 


Let  xvo^  be  an  initial  approximation.  We  place  a  mass  m^ 
of  the  point  x^0^  with  respect  to  the  hyperplane  J  = 


>  0  at  the  reflection 
1,  . . . ,  n.  For  the 


next  iteration  we  take  the  centroid  of  the  system  of  n  masses,  and  continue  the 


iterations  in  this  fashion  using  the  same  respective  masses  tn^.  Algebraically 
this  algorithm  can  be  written  in  the  form 
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,00 


n  /  (k-i), 

(k-l)  2  Y  /<r1*  X  '  ~  yiN 

‘  '  ^  A  ®i  \  ,|_  113  J  ri  * 


i=l 


6 

where  ^  =  £  V  Setting  B  =  (|fH)nXn,  «  -  (6lj»i)nXn,  where  «  is  the 


kPl 

Kronecker  delta,  we  have 


,00  .  ,0.-1)  .  i  A*  „  ,  U  ,(k-l)  ,y3 

■  Cl  ■  ■  A*  ii  i  i]  *0t-l)  t2,*sl(i 

Votruba  [47]  has  shown  that  if  rank  A  >  1  and  m^  ■  HrJ)8,  1  £  i  £  n,  then  the 
sequence  {x^}  converges  to  (I  -  P)  +  A+y,  where  P  is  the  orthogonal 

projection  on  N(A)'1. 

One  can  extend  Citnmino's  method  to  Fredholm  equations  of  the  first  kind 


r° 

(4.1)  Rx  =  Ja  K(*.  t)  x(t)  dt  =  y  ,  ye  LjCa,  b], 
when  K(s,  t)  e  b]  X  [a,  b]},  by  defining  the  family  of  hyperplanes 


Hb  ■  {x  e  Ljta,  b]  :  K(s,  t)  x(t)  dt  =  y(s; 


)} 
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for  almost  every  s  e  [a,  b].  Then  the  orthogonal  projection  of  a  function 
x  e  L,[a,  b]  into  the  hyperplane  H  is  given  by  the  function 

O  2  8 


and  the  reflection  of  x  through  H  is  given  by  x  +  2X(s)  K(s,  •)• 

OS  o 

We  first  note  that  z  e  H  .  Indeed, 

8  s 


<ib  pb  pb 

Ja  K(s,  t)  zg(t)  dt  =  Ja  K(s,  t)  xo(t)  dt  +  X(s)  Ja  K(s,  t)  K(s,  t)  dt 


rb  y(s)  -  f  K(s,  r)  x  (r)  dr  pb 

*>  *'>  “  +  J>C,r>l^ -  J*  |K<S-  £>l 


y(»). 


Thus  for  almost  all  s  e  [a,  b],  t  c  H  .  Now  we  show  that  x  -  z  is  orthogonal 

8  8  0  8 

to  z  -  z  for  all  z  e  H 

a  s 


<XQ  -  za,  za  -  z>  =  <-X(s)  K(s, •) ,  XQ  +  X(s)  K(s,-)  -  z> 


y  -  [  K(s,  r)  x  (r)  dr  pb 

X(s)  {<K(s,  •),  x  >+ sfr - 7T -  Ja  |K(s,  r)|adr 

0  J I  |k(s,  r)|3dr  Ja 


"Ja  K*8»  dr5 


=  0  for  z  6  H  . 
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The  next  iterete  x^  in  the  domino  iteration  would  be  the  centroid  of  the 
family  of  points  with  the  appropriate  weight  functions.  That  is,  letting  m(a)  - 
£  |K(s,  t)|a  dt  be  the  mass  density  and  3  «  JlfJ  |K(a,  t)|a  dt  da  be  the  total 
mass,  we  get 


x1(t)  = 


®(s)  [xQ(t)  +  2  X(s)  K(s,  t) 3  ds 


Xo(t)  "  f 


j!  k<b»  *>  j!  *<®. 


xo(r> 


dr  ds 


+  f  Ja  K^“*  ^  dB* 

that  is, 

xl(t)  c  (I  ‘  f  *0<t)  +  f  <***><*> » 

and  in  general, 

(4‘3)  XiH-l(t)  =  (I  •  f  **«>  *n(t>  +  f  <***><*>  • 

We  note  that  (4.3)  is  a  particular  realization  of  the  successive  approximation 
scheme  with  an  averaging  parameter, 


(*•*)  x  ..  *■  x  -  or  «*Rx  +  a  ft*y 

nr  i  n  n 

it  it 

for  a  solution  of  the  normal  equation  ft  ftx  ■  ft  y,  or  equivalently,  a  least 
squares  solution  of  (4.1)  if  such  a  solution  exists.  The  parameter  or  is  a 


-24- 


#111? 


prescribed  number  in  this  case.  It  should  be  pointed  out,  however,  that  one 
cannot  apply  the  convergence  theorems  for  iterative  methods  for  singular  linear 
operator  equations  developed  by  Keller  [26],  Petryshyn  [35],  and  Votruba  [47] 
since  the  range  of  ft  is  not  closed  unless  ft  is  degenerate.  For  the  same  reason, 
(4.1)  need  not  have  a  best  approximate  solution  for  each  y  e  I^ta,  b].  We  shall 
show  that  for  each  y  e  p(ft^),  the  sequence  (4.3)  converges  to  the  best  approximate 
solution  of  (4.1),  provided  dim  R(ft)  >  1.  The  proof  relies  on  a  series  representa¬ 
tion  for  the  generalized  inverse  of  a  bounded  linear  operator  with  arbitrary  range 
and  on  the  norm  inequality  given  in  Proposition  4.3.  We  now  digress  to  discuss 
such  representations  and  convergence  of  the  successive  approximation  method  to 
best  approximate  solutions  of  linear  integral  equations. 

Let  T  :  X  •*  Y  be  a  bounded  linear  operator  and  X,  Y  be  two  Hilbert  spaces. 
Assume  that  N(T)  ^  [o]  and  let  a  be  any  poaitive  real  number.  Then  (I  -  a  T  T)n 
converges  to  I  -  P,  (P  is  the  projection  on  N(T)X)  in  the  operator  norm  if  and 
only  if  R(T)  is  closed  and  0  <  or  <  U^ja  (8«e  Petryshyn  [35]) . 

2 

The  optimal  value  of  or  is  crb  =  jLip  ,  where 


Y  ■=  g.l.b  {  ^  :  x  e  N(T)\  x  t  0} 
with  the  error  estimate 


IIt^II 


11(1  -  Qfb  T 


where  c  -  ||t||  Ht^I  is  the  pseudo-conditional  number  of  T. 

Furthermore,  if  T  is  closed,  one  easily  obtains  a  Neumann-type  series  expansion 
for  Tf, 
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(4.5)  T*  .  I  (I  -  «  T*T>°  «  T*.  0  <  «  <  , 

r*=0 

(see  C 3A] ,  [47],  [2]). 

We  consider  the  successive  approximations 

(4.6)  xn  =  (I  -  Of  T*T)  xn_1  +  Of  T*y  . 

*  f  •ff  ^ 

Using  the  relation  TTTy=TQy=Ty,  it  follows  by  recursion  that 

t  *  *  +  f 

x  -  T  y  =  (I  -  Of  1  I  x  ,+aTTTy-Ty 
n  n-1  J  1 

-  (I  -  Of  T*T)  (xn_x  -  Tfy) 

it  ^ 

(I-ffII)”  (xo  -  ly). 

Hence, 

lim  (xn  -  Tfy)  =  lim  (I  -  Of  T*T)n  (xq  -  T+y) 
n-*®  n-*® 

-  (I  -  P)  (xo  -  T+y)  -  (1  -  P)  xo, 

since  T  y  €  f?(T  ).  Thus  we  have 

2 

Proposition  4.1.  For  0  <  Of  <  jj^jjsr  »  the  sequence  {xn}  defined  by  (4.6)  converges, 
for  any  Initial  approximation  xq,  to  T^y  +  (I  -  P)  xo>  which  is  a  best  approximate 
solution  of  Tx  s  y.  in  particular  the  optimal  choice  a  »  er^  yields 
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l|Tf,  ♦  (I  -  P)  x,  -  xj  X  {  l*o  ‘  TtHI- 

Corollary  4.1.  Let  11  be  a  completely  continuous  operator  on  a  Hilbert  space  H  into  H, 

and  1  /  0  be  a  eigenvalue  of  R.  Then  the  sequence  (4.6)  with  T  =  R  -  Xl  and 
2 

0  <  or  <  j|-jp  converges  to  a  best  approximate  solution  of  the  linear  equation 
of  the  second  kind  (3.4). 

★  n 

Since  convergence  of  (I  -  a  T  T)  in  the  operator  norm  to  I  -  P  is  equivalent 

★  n 

to  the  range  of  T  being  closed,  and  since  pointwise  convergence  of  (I  -  a  T  X) 
would  be  adequate  to  establish  the  convergence  of  {xn) ,  it  is  natural  to  seek 
conditions  under  which  (1  -  or  T*T)n  converges  pointwise  when  R(T)  is  not  closed. 
For  each  x  e  X,  we  have 


lim  (I  -  a  T*T)nx  = 

n=«co 


(I  -  P)x,  for  0  <  Of  < 


where  R(T)  is  not  necessarily  closed.  (See  Showalter  and  Ben-Israel  [4l].)  The 
series 


m 

Z*  v  * 
(I  -  or  T  T)  or  T  y 

te=0 


for 


0  <  of  < 


w 


converges  in  norm  monotonically  to  T^y  for  any  y  e  P(T^)  =  R(T)  +  R(T^).  (Compare 
with  (4.5)  for  the  case  of  a  closed  range.)  Moreover,  if  Qy  c  R(TT  ),  then 


||Tfy  -  Y  (1  -  Of  T*T)k0f  T*y|)a  s  - - * 

||(TT  )Tyll  + 


3  "(ttVyII3 


(KTT^J^y  II Z+  n«( 2- a  ||T  || 2)  ||Tty  ||* 


#1117 


-Zl- 


Rephrased  in  Che  setting  of  the  iterative  process  (4.6),  the  above  expansions 
yield  easily  the  following  proposition. 

Proposition  4.2.  Let  X  and  Y  be  two  Hilbert  spaces  over  the  same  field  and  T 
be  a  bounded  linear  operator  on  X  into  Y,  with  the  range  of  T  not  necessarily 
closed.  The  sequence  (4.6)  starting  with  xq  =  0,  converges  in  norm  monotonically 

to  T%  whenever  y  e  P(T^)  =  R(T)  +  R(T)  and  a  is  any  fixed  number  in  the  range 

2  * 

0  <  a  <  Tfrfs.  Moreover,  if  Qy  e  R(TT  ),  then 


U^yll8  j|  (TT*)+y||a 

||(TT*)ty||a  +  n  or  (2  -  or||T||a)||  T+y|I* 


We  now  return  to  the  consideration  of  convergence  of  the  generalization 
of  Cimmino's  method  for  integral  equations  of  the  first  kind. 

Proposition  4.3.  Let  ft  be  the  integral  operator  defined  by  (4.1),  where  K(s,  t) 
e  L2([a,  b]  X  [a,  b]}.  If  the  dimension  of  the  range  of  R  is  greater  than  one, 
then 


Proof.  We  will  first  show  that  dim  r(R)  >  1  implies  dim  R(R  R)  >  1.  For  if  dim 
R(ft*ft)  s  1,  then 


R(ft*ft)X 


N(ft*  ft) 


M(R) . 
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Thus  the  deficiency  of  W(ft)  is  not  greater  than  one  and  dim  R(R)  £  1. 
It  is  well  known  that 


(A.  7)  p<R*fr)  «  ||**al|  =  ||j»J|a  *  „ 

■Hr  ^ 

where  p(ft  ft)  denotes  the  spectral  radius  of  ft  ft.  (See,  for  instance,  [43].) 

Let  >  \2  2  ...  a:  0  be  the  eigenvalues  of  R*ft  with  a  corresponding  orthonormal 

rb 

set  of  eigenfunctions  0Jf  02>  ....  and  let  M(s,  t)  =  Ja  K(u,  s)  K(u,  t)  du.  Then 
by  Bessel's  inequality, 

K  0t(8)  E  fa  M<8»  V°  dt  <  tM(8»  t) ]adt. 


Note  that  the  strict  inequality  in  Bessel's  inequality  holds  here  since  r(R*R)  > 
1.  Integrating  both  sides  of  the  preceding  inequality,  we  get 


s  <  £  J><-  -  ■  f.  f.  [/!  K(u,  b)  K(u,  t)  du]3  dt  ds 

« f.  f.  {/!  [K(u,  s)]3  du  [K(u,  t)]3  du}  dt  ds 

-(£  r.  ■■■•.  s)f  du  d|«  . 


Thus  X^  <  B.  Thus  from  (4.7)  we  have  |]r||3  <  3  since  X^  -  p(ft*R). 

Theorem  4.1.  if  dim  Rfll)  >  1,  then  the  generalized  method  of  Cimmino  converges 
■onotonlcally  to  a  best  approximate  solution  of  minimal  norm  of  the  integral 
equation  of  the  first  kind,  starting  from  the  initial  approximation  xq  -  0,  for 
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any  y  e  {)(«*)  =  R(fl)  +  R(R)'L 


and 


11  pTy|[a  ll(»»Vy||a 

IU«*Vy||*  +  p  <9  -  MP>  ||*T3r|C  ' 


,b  «b 


where  &  =  Ja  Ja  [K(u,  a)  I 


du  ds. 


Proof.  The  theorem  Is  a  consequence  of  Propositions  4.2  and  4.3. 

Let  K(s,  t)  be  a  symmetric  positive  definite  kernel  and  assume  that  the 

integral  equation  (3.1)  is  solvable.  Fridman  [ll]  has  shown  that  fir  any  or  in 

the  interval  0  <  a  <  2or  ,  where  or  is  the  smallest  characteristic  value  of  the 

l  l 

kernel  K(s,  t) ,  then  the  sequence 


x^Cs)  =  xn(s)  +  Of[y(s)  -  ftxn(t)] 


converges  in  the  norm  of  L^Ca,  b]  to  the  solution  starting  from  any  initial 
approximation  xq  e  Ljta,  b],  (See  also  Hikhlin  and  Smolitskiy  [31].)  Bialy  [4] 
generalized  Fridman's  result  and  proved  the  following  theorem.  Let  A  be  a 
bounded  linear  operator  on  a  Hilbert  space  H  into  H,  and  suppose  also  that  A 
is  nonnegative:  (Ax,  x)  s  0  for  all  x  e  H.  For  y  e  H,  xq  s  H,  consider  the 
iterative  process 


Vl  "  xn  +  “(y  -  AVl> 


where  0  <  or  <  2  |] A  ||  .  Then  Axn  Qy,  whete  Q  is  the  orthogonal  projection  on 
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R(A) .  {k  )  converges  if  end  only  if  Che  equation  Ax  =  y  has  a  solution,  in 

*  A 

which  case  x  -»  (I  -  P)x  +  x,  where  x  is  the  solution  of  minimal  norm.  Related 
n  o 

results  on  iterative  methods  for  the  solutions  of  nonnegative  operators  were 
obtained  by  Keller  [26].  Proposition  4.2  generalizes  the  results  of  Fridman 
and  Bialy  to  the  setting  of  best  approximate  solutions  and  expresses  the  results 
and  error  bounds  in  terms  of  generalized  inverses. 


#1117 


-31- 


5.  Steepest  Descent  and  Conjugate  Gradient  Methods  for  Best  Approximate 


Solutions  of  Linear  Integral  Equations  ♦ 

We  consider  first  the  Integral  equation  of  the  second  kind  (3.4). 

Kantorovich  [24]  has  shown  that  if  K(s,  t)  is  synmetric  and  X  <  X^,  k  =  1,  2,  ..., 
where  X^  are  the  characteristic  values  of  the  kernel  K(s,  t),  then  the  method 
of  steepest  descent  for  the  solution  of  (34),  i.e.,  the  sequence 


converges  to  the  unique  solution  x  of  (3.4).  The  speed  of  convergence  is 
determined  by 


where  m  =  Tnf*  -r~,  m  »  m,in  and  c  is  a  constant.  This  result  follows  from 
i  k  \  a  k  Xfc 

a  direct  application  of  the  well  known  theorem  of  Kantorovich  [24,  25]  on  the 
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convergence  of  the  method  of  steepest  descent  for  positive  definite  bounded 
linear  operators  on  a  Hilbert  space.  See  also  Hayes  [52]  for  related  results. 

When  m  »  1,  (3.4)  of  course  may  not  have  a  solution.  However,  if  a 
solution  exists,  then  the  sequence  of  steepest  descent  converges  to  it. 
Kantorovich's  theorem  does  not  apply  to  integral  equations  of  the  first  kind 
with  nondegenerate  kernel. 

In  this  section  we  extend  the  applicability  of  the  method  of  steepest 
descent  and  the  conjugate  gradient  methods  to  integral  equations  with  nonunique 
solutions  and  to  integral  equations  of  the  first  kind.  Convergence  of  these 
methods  in  the  mentioned  settings  will  be  asserted  using  recent  results  of 
the  authors  [22],  [23],  [32]  on  singular  linear  operator  equations. 

Let  T  be  a  bounded  linear  operator  on  a  Hilbert  space  X  into  a  Hilbert 
space  Y.  The  method  of  steepest  descent  for  minimizing  the  functional  J(x)  = 

II Tx  -  y||3  for  y  e  Y  is  defined  by  the  following  sequence  starting  with  an 
initial  approximation  xq 

(5.D  x^-  xn-ornrn  n  .  0,  1,  ... 

where 

(5.2)  rn  -  T*(Txn  -  y) 

and 


The  following  theorem  is  an  immediate  consequence  of  [32]  using  (3.9)  -  (3.10) 
and  Theorem  3.1. 
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Theorem  5.1.  Let  T  =»  I  -  Xft,  where  ft  la  aa  In  Section  3.  Then  the  8equence 
{xj  of  steepest  descent  defined  by  (5.1)  -  (5.3)  with  any  initial  approximation 
c  LjCat  b]  converges  in  Ljta,  b]  to  the  best  approximate  solution  (I  -  Xft)^y  + 
(1  -  P)xq  of  the  integral  equation  of  the  second  kind  (3.4)  for  each  y  e  LjCa,  b], 
and 


||(I  -  Aft)+y  +  (I  -  P)xo 


9 


where  C  is  a  constant  and 


m||x||3  *  <(I  -  Xft)(I  -  Xft*)x,x>  £  M||x||a,  x  e  R(I  -  Xft*). 

The  sequence  of  steepest  descent  can  be  written  in  this  case  in  the  form 

x^^a)  =  (1  -  on)  xn(»)  -  «n  CxJa  k(a,  t)  x(t)  dt  -  g(s)], 

where  k(s,  t)  and  g(s)  are  defined  in  (3.10). 

The  sequence  {x  }  in  the  above  theorem  converges  in  the  mean.  However,  a 
n 

sequence  may  be  constructed  using  {x  }  that  converges  uniformly  to  (I  -  Xft)  y  + 


(I  -  P)  xq.  Indeed,  if  we  define 


zR(s)  =  g(s)  +  Xja  k(a,  t)  xn(t)  dt. 


then  {z  (s)}  converges  uniformly  to  a  beat  approximate  solution  of  (3.4). 
n 


-34- 


#1117 


For  integral  equations  of  the  first  kind  (3.3)  we  have  the  following 
theorem  as  a  consequence  of  Theorem  3.2  and  Theorem  3.2  in  [22]. 

* 

Theorem  5.2.  Let  T  =  ft  as  defined  in  Section  3.  If  Qy  e  R(RR  ),  then  the 
sequence  [x^]  of  steepest  descent  defined  by  (5.1)  -  (5.3)  with  initial  approximation 
xq  =  0  converges  monotonically  to  R^y,  the  best  approximate  solution  of  minimal 
norm  of  the  integral  equation  of  the  first  kind  (3.3),  and 

,  Ml*  IteV  ||«sVy|f 

llx  -  R  y||a  £  - j-i - + - 

"  Iter  lies* Vy|f  +  »  II*  V 

for  n  =  1,  2,  ••*. 

We  now  consider  the  conjugate  gradient  method  of  Hestenes  and  Stiefel  [17],  [52] 
for  minimizing  the  functional  J(x)  =  ||Tx  -  y||a .  We  let  =  pQ  =  T  (Txq  -  y) 
and  if  p  ^  0,  then  compute  x^  =  x  -  a  p  ,  where  a  ~  ^ro^  .  For  n  =  1,  2,  ... 

llTpJP 

compute 
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(5.7) 


The  following  theorem  is  a  consequence  of  Theorem  3.1, and  Theorem  4.1  in  [23]. 

Theorem  5.3.  Let  T  =  I  -  Xft  where  ft  is  as  in  Section  3.  Then  the  sequence 

[x  }  generated  by  the  conjugate  gradient  method  (5.4)  -  (5.7)  converges 
n 

monotonically  starting  from  any  initial  approximation  xq  e  b]  to  the  best 

approximate  solution  u  =  (I  -  Xft)^y  +  (I  -  P)xq  of  the  integral  equation  of  the 
second  kind  (3.4)  for  each  y  e  LjCa,  b],  and 


where  m  and  M  are  the  spectral  bounds  of  the  restriction  of  the  operator 
(I  -  Xft*) (I  -  Xft)  to  R(I  -  Xft*). 

As  a  consequence  of  Theorem  3.2  ,  and  Theorem  5. 1  in  [23]  we  have  the  following 
theorem  for  integral  equations  of  the  first  kind. 

Theorem  5.4.  Let  T  =  ft  as  defined  in  Section  3.  If  Qy  e  R(RRR),  then  the 

* 

conjugate  gradient  method  (5.4)  -  (5.7)  with  initial  approximation  xq  e  R(R  R), 
converges  monotonically  to  the  best  approximate  solution  of  minimal  norm  of  the 
integral  equation  of  the  first  kind  (3.3)  and 
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The  theorems  on  the  convergence  of  the  successive  approximation  method, 
the  steepest  descent  and  the  conjugate  gradient  methods  also  apply  to  oest 
approximate  solutions  of  Volterra  linear  integral  equations  of  the  first  and 
second  kinds. 

6.  Bibliographical  Comments. 

There  is  an  extensive  literature  on  the  steepest  descent  and  the  conjugate 
gradient  methods  for  linear  operator  equations,  going  back  to  the  papers  of 
L.  V.  Kantorovich  [24]  and  R.  M.  Hayes  [52],  respectively,  and  to  the  recent 
work  of  the  authors  [32],  [22],  [23]  on  singular  operator  equations.  For  various 
contributions  to  these  methods  for  linear  operator  equations ,  we  refer  the  reader 
to  the  bibliographical  comments  made  in  [22],  [23];  the  latter  reference  also 
contains  an  extensive  bibliography  on  the  conjugate  gradient  method  and  related 
variants  for  linear  and  nonlinear  operator  equations  and  minimization  problems 
in  various  settings.  In  the  present  paper  we  have  confined  our  bibliography 
to  relevant  references  dealing  with  integral  equations  and  related  aspects  of 
generalized  inverses  and  iterative  methods.  For  other  contributions  to  generalized 
inverses  oFlinear  operators  and  related  topics  not  considered  here,  see  Nashed  [53]. 
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